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1  Objectives 

Means  to  build  fault-tolerant  services  have  been  at  hand  for  some  time.  De¬ 
fense  against  attacks  remains  a  difficult  problem,  though.  And  this  problem 
becomes  ever  more  urgent  with  the  increasing  use  of  networked  computing 
systems  in  our  society’s  critical  infrastructures  and  in  future-generation  mil¬ 
itary  systems  (such  as  GIG  and  JBI).  The  objective  of  this  AFOSR-funded 
effort  is  to  bridge  the  gap  from  fault-tolerance  to  attack-tolerance  by  explor¬ 
ing  two  threads. 

•  The  use  of  mechanically-generated  diversity  for  creating  independent 
server  replicas  and  a  “moving  target”  defense. 

•  Language-based  techniques  to  build  a  new  theoretical  basis  for  autho¬ 
rization  and  for  quantifying  information  flow  and  information  corrup¬ 
tion. 

2  Summary  of  Completed  Research 

2.1  Moving  target  defenses:  Theory  and  practice 

Semantic  Framework  for  Diversity.  A  set  of  replicas  is  diverse  to  the 
extent  that  all  implement  the  same  functionality  but  differ  in  their  irnple- 
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mentation  details.  Diverse  replicas  are  less  prone  to  having  vulnerabilities 
in  common,  because  attacks  typically  depend  on  memory  layout  and/or 
instruction-sequence  specifics. 

Recent  work  advocates  using  mechanical  means,  such  as  program  rewrit¬ 
ing,  to  create  such  diversity.  A  correspondence  between  the  specific  trans¬ 
formations  being  employed  and  the  attacks  they  defend  against  is  often 
provided.  But  little  has  been  said  about  the  overall  effectiveness  of  diversity 
per  se  in  defending  against  attacks.  With  this  broader  goal  in  mind,  we  de¬ 
veloped  a  precise  characterization  of  attacks,  applicable  to  viewing  diversity 
as  a  defense.  In  addition,  we  showed  how  mechanically-generated  diversity 
compares  to  a  well-understood  defense:  strong  typing. 

The  reduction  we  derived — defenses  created  by  mechanically-generated 
diversity  are  forms  of  probabilistic  dynamic  type-checking — was  surprising. 
Unfortunately,  this  result  ignores  probabilities,  which  do  matter  for  practical 
application.  The  work  is  thus  best  seen  as  only  a  first  step  in  characterizing 
the  effectiveness  of  program  obfuscation  and  other  forms  of  mechanically- 
generated  diversity. 

Proactive  Obfuscation  Prototype.  Proactive  obfuscation  is  a  moving- 
target  defense.  It  involves  running  diverse  replicas  and  periodically  selecting 
a  replica  for  replacement  by  a  new  one,  where  that  new  one  differs  internally 
from  any  prior  replica  used  in  the  system.  Proactive  obfuscation  creates  in¬ 
dependence  and  also  preserves  it  over  time,  even  when  attackers  can  analyze 
individual  replicas. 

A  firewall  controls  the  passage  of  packets  from  some  outer  network  to 
an  enclave  it  is  protecting.  Because  it  resides  at  the  border  to  a  potentially 
hostile  network,  attack-tolerance  is  crucial  for  a  firewall.  And  because  it 
is  the  sole  means  by  which  packets  enter  or  exit  an  enclave,  availability  is 
also  important.  A  firewall  was  thus  an  ideal  service  to  implement  using 
an  approach,  like  proactive  obfuscation,  intended  for  building  trustworthy 
services. 

We  built  that  prototype  and  conducted  a  rather  extensive  analysis  of  its 
performance.  This  analysis  involved  deploying  different  implementations  of 
the  various  underlying  mechanisms,  so  that  we  could  identify  bottlenecks. 
We  also  implemented  a  distributed  storage  service  that  uses  Byzantine  Quo¬ 
rum  Systems  (rather  than  state  machine  replication)  and  employs  proactive 
obfuscation  to  create  the  artificial  diversity  needed  for  independence  among 
those  servers. 
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Convincing  Alternative  to  Digital  Signatures.  Systems  that  employ 
proactive  obfuscation  and,  indeed,  most  protocols  for  fault-tolerance  and 
attack-tolerance,  often  rely  on  public-key  signatures  for  message-source  au¬ 
thentication,  defense  against  corruption  of  in-transit  messages,  and  also  as 
a  proof  to  recipients  of  correct  protocol  execution.  In  our  proactive  obfusca¬ 
tion  prototypes,  as  in  many  distributed  services,  the  overhead  of  generating 
and  checking  signatures  must  be  kept  low.  With  public-key  signatures  it 
won’t  be,  which  prompted  us  to  develop  lower-cost  alternatives  for  this  set¬ 
ting. 

A  k- convincing  tag  can  be  forwarded  at  least  k— 1  times  and  still  be  guar¬ 
anteed  to  convince  receivers  of  its  authenticity.  We  developed  an  O(knlogn) 
probabilistic  protocol  for  the  case  when  n  hosts  might  receive  such  a  tag; 
in  some  contexts,  our  protocol  is  faster  than  using  public- key  signatures. 
We  also  developed  an  0{n2  log2n )  probabilistic  protocol  for  computing  k- 
convincing  tags  for  all  k  simultaneously.  And  we  ran  experiments  to  compare 
the  performance  of  these  protocols  to  fast  implementations  of  public-key  sig¬ 
natures  and  to  closely  related  MAC  constructions. 

2.2  Language-based  techniques 

Hyperproperties.  Important  classes  of  security  policies  cannot  be  ex¬ 
pressed  using  what  have  been  termed  properties,  sets  of  execution  traces  for 
which  membership  of  a  trace  depends  on  the  trace  alone  and  not  on  which 
other  traces  are  in  the  property.  For  example,  noninterference  is  a  confiden¬ 
tiality  policy  that  stipulates  commands  executed  on  behalf  of  users  holding 
high  clearances  have  no  effect  on  system  behavior  observed  by  users  with 
only  low  clearances.  Noninterference  is  not  a  property,  because  whether 
some  given  trace  is  allowed  depends  on  whether  another  trace  (obtained  by 
deleting  command  executions  by  high  users)  is  allowed.  As  a  second  ex¬ 
ample,  stipulating  a  bound  on  average  response  time  over  all  executions  is 
an  availability  policy  that  cannot  be  specified  as  a  property,  because  the 
acceptability  of  delays  in  any  given  execution  depends  on  the  magnitude  of 
delays  in  all  other  executions. 

These  expressiveness  limitations  are  overcome  by  using  a  new  abstraction 
we  developed — hyperproperties ,  sets  of  properties  (i.e. ,  sets  of  sets  of  traces). 
There  are  two  interesting  classes  of  hyperproperties:  safety  and  liveness. 
And  we  have  been  able  prove  the  following. 

•  Hyperproperties  can  describe  properties  and,  moreover,  can  describe 
security  policies,  such  as  noninterference  and  average  response  time, 
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that  properties  cannot.  Indeed,  we  have  not  been  able  to  find  require¬ 
ments  on  system  behavior  that  cannot  be  specified  as  a  hyperproperty. 
Deterministic,  non  deterministic,  and  probabilistic  system  models  all 
can  be  handled  using  hyperproperties. 

•  Every  hyperproperty  is  the  intersection  of  a  safety  hyperproperty  and 
a  liveness  hyperproperty.  Safety  hyperproperties  and  liveness  hyper¬ 
properties  thus  form  a  fundamental  basis  from  which  all  hyperproper¬ 
ties  can  be  constructed. 

•  The  topological  characterization  of  properties  can  be  generalized  to 
characterize  hyperproperties,  and  the  result  is  equivalent  to  the  lower 
Vietoris  topology. 

We  have  not  been  able  to  obtain  complete  verification  methods  for  safety 
hyperproperties  or  for  liveness  hyperproperties,  but  we  have  been  able  to 
generalize  prior  work  on  using  invariance  arguments  to  verify  information- 
flow  policies.  Our  generalization  is  applicable  to  a  class  of  hyperproperties 
we  introduce  called  k -safety. 

We  have  also  been  able  to  relate  the  hyperproperties  framework  to  step¬ 
wise  refinement.  Whereas  safety  properties  are  preserved  by  such  refinement 
steps,  hypersafety  properties  are  not.  We  also  explored  the  extent  to  which 
the  hyper  proper  ties  framework  applies  to  arbitrary  system  representations, 
such  as  relations,  labeled  transition  systems,  and  probabilistic  state  ma¬ 
chines. 

Quantification  of  Integrity.  Hyperproperties  are  qualitative.  In  fact, 
the  usual  characterization  of  security  in  terms  of  confidentiality,  integrity, 
and  availability  is  qualitative.  Engineering  realities  often  require  quantita¬ 
tive  characterizations.  Methods  have  long  existed  for  specifying  and  verify¬ 
ing  quantitative  bounds  on  the  flow  of  confidential  information.  Yet  methods 
for  quantification  of  corruption — that  is,  damage  to  integrity — have  received 
little  attention  to  date. 

Under  the  auspices  of  this  funding,  we  developed  a  framework  and  a 
method  to  calculate  bounds  on  integrity  corruption.  To  quantify  corrup¬ 
tion,  a  formal  definition  of  “integrity”  was  required.  We  took  two  distinct 
notions  of  information  modification  as  points  of  departure:  taint  analysis 
and  program  correctness.  These,  in  turn,  led  to  two  distinct  measures  of 
corruption  that  we  named  contamination  and  suppression. 

Contamination  is  defined  to  be  the  flow  of  information  from  untrusted 
inputs  to  outputs  that  are  supposed  to  be  trusted.  Trusted  outputs  are 
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not  supposed  to  be  influenced  by  untrusted  information,  so  contamination 
corrupts  integrity.  Flow  between  untrusted  and  trusted  objects  was  first 
studied  by  Biba,  who  identified  a  duality  between  models  of  integrity  and 
confidentiality.  The  confidentiality  dual  to  contamination  is  leakage,  which 
is  information  flow  from  secret  inputs  to  public  outputs.  The  Biba  duality 
thus  suggests  that  any  method  for  measuring  leakage  of  information  could 
serve  as  the  basis  for  measuring  contamination. 

Our  other  measure  for  corruption,  suppression,  is  derived  from  program 
correctness.  For  a  given  input,  a  correct  implementation  should  produce 
an  output  o  permitted  by  a  specification.  Any  knowledgeable  user  of  these 
implementations  could  recover  o  from  the  implementation’s  output.  With 
programs  and  channels,  suppression  occurs  when  information  is  lost.  Infor¬ 
mation  theory  can  be  used  to  quantify  suppression,  including  how  to  bound 
the  attacker’s  influence  on  suppression. 

We  might  suspect  that  contamination  generalizes  suppression,  or  vice 
versa,  but  we  have  proved  this  is  not  the  case.  Moreover,  we  have  been  able 
to  use  our  approach  to  derive  quantitative  measures  for  various  anonymiza¬ 
tion  algorithms  that  have  been  proposed  to  support  database  privacy:  k- 
anonymity,  /-diversity,  and  differential  privacy.  This  work  is  based  on  a 
new  theorem  we  derived;  it  asserts  that  suppression  plus  leakage  is  neces¬ 
sarily  a  constant  (related  to  the  information  content  of  the  object  being 
anonymized) . 

Credentials-based  Authorization.  Authorization  is  fundamental  to  im¬ 
plementing  any  trustworthy  system.  To  be  trustworthy,  a  system  must  be¬ 
have  as  expected  but  not  exhibit  any  other  behaviors.  And  authorization, 
which  governs  what  requests  a  system  will  accept,  is  the  way  requests  for 
unacceptable  system  behaviors  are  blocked. 

Nexus  Authorization  Logic  (NAL)  was  developed  to  provide  a  princi¬ 
pled  basis  for  specifying  and  reasoning  about  credentials  and  authorization 
policies.  It  extended  prior  access  control  logics  based  on  “says”  and  “speaks- 
for”  operators,  enabling  within  a  single  framework  request  authorization  to 
depend  on  (i)  the  source  or  pedigree  of  the  requester,  (ii)  the  outcome  of 
performing  an  analysis  on  the  requester,  or  (iii)  the  use  of  trusted  software 
to  encapsulate  or  modify  the  requester.  Prototype  document-viewer  appli¬ 
cations  that  enforce  integrity  and  confidentiality  of  document  contents — all 
implemented  on  the  Nexus  operating  system — have  been  built  to  illustrate 
the  convenience  and  expressive  power  of  this  approach  to  authorization. 
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3  Impacts  on  the  Community 

It  can  be  difficult  to  trace  the  impact  that  ideas  have,  and  the  research 
reported  herein  is  mostly  concerned  with  new  ideas.  However,  there  are  two 
direct  consequences  that  can  likely  be  traced  to  our  research.  They  concern 
framing  of  the  problem  space  (and  ultimately  new  federal  funding). 

•  The  CNCI  “moving  target  defense”  focus  area  apparently  had  its  ori¬ 
gins  with  the  proactive  obfuscation  work  discussed  above. 

•  A  MURI  and  a  new  CNCI  initiative  on  science  of  security,  including  a 
DDRE-funded  Jasons  study  (JSR-10-102)  in  Summer  2010,  all  are  di¬ 
rect  responses  to  the  Pi’s  advocacy  (in  Congressional  testimony  and  in 
discussions  with  DoD).  The  language-based  technique  discussed  above 
has  provided  examples  of  the  pay-off  these  investments  could  provide. 

There  are  other  less-direct  transitions  that  arise  through  the  Pi’s  in¬ 
volvement  in  various  advisory  capacities. 

•  Schneider  is  Chief  Scientist  of  the  NSF  TRUST  Science  and  Technol¬ 
ogy  Center,  which  includes  U.C.  Berkeley,  Carnegie-Mellon  University, 
Cornell  University,  Stanford  University,  and  Vanderbilt  University. 

•  Schneider  is  a  member  of  the  following  industrial  advisory  boards: 
Fortify  Software  Technical  Advisory  Board;  Microsoft’s  Trustworthy 
Computing  Academic  Advisory  Board  (co-chair). 

•  Schneider  served  on  the  following  other  advisory  committees:  NIST  In¬ 
formation  Security  and  Privacy  Advisory  Board;  Computing  Research 
Association  Board  of  Directors;  Defense  Science  Board;  Computing 
Community  Consortium  Council. 
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